erofs-utils: introduce fuse implementation
A message that is not in the archive (sent off-list or not yet received): [email protected]
cover erofs-utils: introduce fuse implementation
Hi all,
This patchset is based on the original patch
https://lore.kernel.org/r/[email protected]
and several fixes / compression support by Huang Jianan
https://lore.kernel.org/r/[email protected]
with my additional 9 incremental patches to refactor my current development status, which will be folded into original patches in the next WIP version since the fuse approach hasn't been merged into dev branch yet.
To summarize the benefits of erofsfuse, I think it would be:
- erofs images can be supported on various platforms;
- an independent unpack tool can be developed based on this;
- new on-disk feature can be iterated, verified effectively.
Any feedback or/and follow-on development/cleanup is welcomed. (we still have a lot to do for the entire erofsfuse codebase...)
Thanks,
Gao Xiang
Gao Xiang (9):
erofs-utils: fuse: adjust larger extent handling
erofs-utils: fuse: use proper expression about inode size
erofs-utils: fuse: drop ofs_out
erofs-utils: fuse: refuse a undefined shifted cluster behavior
erofs-utils: fuse: drop z_erofs_shifted_transform()
erofs-utils: fuse: rename ofs_head and outputsize
erofs-utils: fuse: cleanup erofs_read_data_compression()
erofs-utils: fuse: move up mpage in struct erofs_map_blocks
erofs-utils: fuse: fix up source headers
Huang Jianan (2):
erofs-utils: fuse: support read special file
erofs-utils: fuse: support read compressed file
Li Guifu (1):
erofs-utils: introduce fuse implementation
Makefile.am | 2 +- README | 28 ++- configure.ac | 3 +- fuse/Makefile.am | 18 ++ fuse/decompress.c | 84 ++++++++ fuse/decompress.h | 42 ++++ fuse/dentry.c | 129 ++++++++++++ fuse/dentry.h | 43 ++++ fuse/disk_io.c | 72 +++++++ fuse/disk_io.h | 21 ++ fuse/getattr.c | 65 ++++++ fuse/getattr.h | 15 ++ fuse/init.c | 118 +++++++++++ fuse/init.h | 24 +++ fuse/logging.c | 81 ++++++++ fuse/logging.h | 55 ++++++ fuse/main.c | 171 ++++++++++++++++ fuse/namei.c | 242 +++++++++++++++++++++++ fuse/namei.h | 22 +++ fuse/open.c | 22 +++ fuse/open.h | 15 ++ fuse/read.c | 214 ++++++++++++++++++++ fuse/read.h | 17 ++ fuse/readir.c | 123 ++++++++++++ fuse/readir.h | 17 ++ fuse/zmap.c | 417 +++++++++++++++++++++++++++++++++++++++ include/erofs/defs.h | 16 ++ include/erofs/internal.h | 79 ++++++++ include/erofs_fs.h | 4 + 29 files changed, 2156 insertions(+), 3 deletions(-) create mode 100644 fuse/Makefile.am create mode 100644 fuse/decompress.c create mode 100644 fuse/decompress.h create mode 100644 fuse/dentry.c create mode 100644 fuse/dentry.h create mode 100644 fuse/disk_io.c create mode 100644 fuse/disk_io.h create mode 100644 fuse/getattr.c create mode 100644 fuse/getattr.h create mode 100644 fuse/init.c create mode 100644 fuse/init.h create mode 100644 fuse/logging.c create mode 100644 fuse/logging.h create mode 100644 fuse/main.c create mode 100644 fuse/namei.c create mode 100644 fuse/namei.h create mode 100644 fuse/open.c create mode 100644 fuse/open.h create mode 100644 fuse/read.c create mode 100644 fuse/read.h create mode 100644 fuse/readir.c create mode 100644 fuse/readir.h create mode 100644 fuse/zmap.c
Signature · 1 line
2.24.0
PATCH 1/12 erofs-utils: introduce fuse implementation
From: Li Guifu <[email protected]>
Let's add erofsfuse approach, and benefits are:
- images can be supported on various platforms;
- new unpack tool can be developed based on this;
- new on-disk feature can be iterated, verified effectively.
This commit only aims at reading a regular file. Other file (e.g. compressed file) support is out of scope for now.
--- Makefile.am | 2 +- README | 28 ++++- configure.ac | 3 +- fuse/Makefile.am | 14 +++ fuse/dentry.c | 129 ++++++++++++++++++++++ fuse/dentry.h | 42 ++++++++ fuse/disk_io.c | 72 +++++++++++++ fuse/disk_io.h | 21 ++++ fuse/getattr.c | 64 +++++++++++ fuse/getattr.h | 15 +++ fuse/init.c | 98 +++++++++++++++++ fuse/init.h | 22 ++++ fuse/logging.c | 81 ++++++++++++++ fuse/logging.h | 55 ++++++++++ fuse/main.c | 170 +++++++++++++++++++++++++++++ fuse/namei.c | 227 +++++++++++++++++++++++++++++++++++++++ fuse/namei.h | 22 ++++ fuse/open.c | 22 ++++ fuse/open.h | 15 +++ fuse/read.c | 114 ++++++++++++++++++++ fuse/read.h | 16 +++ fuse/readir.c | 123 +++++++++++++++++++++ fuse/readir.h | 17 +++ include/erofs/defs.h | 3 + include/erofs/internal.h | 37 +++++++ 25 files changed, 1409 insertions(+), 3 deletions(-) create mode 100644 fuse/Makefile.am create mode 100644 fuse/dentry.c create mode 100644 fuse/dentry.h create mode 100644 fuse/disk_io.c create mode 100644 fuse/disk_io.h create mode 100644 fuse/getattr.c create mode 100644 fuse/getattr.h create mode 100644 fuse/init.c create mode 100644 fuse/init.h create mode 100644 fuse/logging.c create mode 100644 fuse/logging.h create mode 100644 fuse/main.c create mode 100644 fuse/namei.c create mode 100644 fuse/namei.h create mode 100644 fuse/open.c create mode 100644 fuse/open.h create mode 100644 fuse/read.c create mode 100644 fuse/read.h create mode 100644 fuse/readir.c create mode 100644 fuse/readir.h
Makefile.am +1 -1
@@ -3,4 +3,4 @@ ACLOCAL_AMFLAGS = -I m4 -SUBDIRS = man lib mkfs +SUBDIRS = man lib mkfs fuse
README +27 -1
@@ -2,7 +2,8 @@ erofs-utils =========== erofs-utils includes user-space tools for erofs filesystem images. -Currently only mkfs.erofs is available. +One is mkfs.erofs to create a image, the other is erofsfuse which +is used to mount a image on a directory mkfs.erofs ----------
@@ -95,6 +96,31 @@ It may still be useful since new erofs-utils has not been widely used in commercial products. However, if that happens, please report bug to us as well. +erofs-utils: erofsfuse +---------------------- +erofsfuse mount a erofs filesystem image created by mkfs.erofs to a directory, and then +It can be listed, read files and so on. + +Dependencies +~~~~~~~~~~~~ +FUSE library version: 2.9.7 +fusermount version: 2.9.7 +using FUSE kernel interface version 7.19 + +How to installed fuse +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +sudo apt-get update +sudo apt-get install -y fuse libfuse-dev + +How to build erofsfuse +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + $ ./autogen.sh + $ ./configure + $ make + +erofsfuse binary will be generated under fuse folder. + Contribution ------------
configure.ac +2 -1
@@ -245,6 +245,7 @@ fi AC_CONFIG_FILES([Makefile man/Makefile lib/Makefile - mkfs/Makefile]) + mkfs/Makefile + fuse/Makefile]) AC_OUTPUT
fuse/Makefile.am new file +14 -0
@@ -0,0 +1,14 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Makefile.am + +AUTOMAKE_OPTIONS = foreign +bin_PROGRAMS = erofsfuse +erofsfuse_SOURCES = main.c dentry.c getattr.c logging.c namei.c read.c disk_io.c init.c open.c readir.c +erofsfuse_CFLAGS = -Wall -Werror -Wextra \ + -I$(top_srcdir)/include \ + $(shell pkg-config fuse --cflags) \ + -DFUSE_USE_VERSION=26 \ + -std=gnu99 +LDFLAGS += $(shell pkg-config fuse --libs) +erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl +
fuse/dentry.c new file +129 -0
129 lines not on this page · raw message
fuse/dentry.h new file +42 -0
42 lines not on this page · raw message
fuse/disk_io.c new file +72 -0
72 lines not on this page · raw message
fuse/disk_io.h new file +21 -0
@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* erofs-fuse\disk_io.h +* Created by Li Guifu <[email protected]> +*/ + +#ifndef __DISK_IO_H +#define __DISK_IO_H + +#include "erofs/defs.h" +#include "erofs/internal.h" + +int dev_open (const char *path); +void dev_close (void ); +int dev_read (void *buf,size_t count,off_t offset); + +static inline int dev_read_blk (void *buf,uint32_t nr) +{ +return dev_read (buf,EROFS_BLKSIZ ,blknr_to_addr (nr)); +} +#endif
fuse/getattr.c new file +64 -0
64 lines not on this page · raw message
fuse/getattr.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* erofs-fuse\getattr.h +* Created by Li Guifu <[email protected]> +*/ + +#ifndef __EROFS_GETATTR_H +#define __EROFS_GETATTR_H + +#include <fuse.h> +#include <fuse_opt.h> + +int erofs_getattr (const char *path,struct stat *st); + +#endif
fuse/init.c new file +98 -0
98 lines not on this page · raw message
fuse/init.h new file +22 -0
@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* erofs-fuse\init.h +* Created by Li Guifu <[email protected]> +*/ + +#ifndef __EROFS_INIT_H +#define __EROFS_INIT_H + +#include <fuse.h> +#include <fuse_opt.h> +#include "erofs/internal.h" + +#define BOOT_SECTOR_SIZE 0x400 + +int erofs_init_super (void ); +erofs_nid_t erofs_get_root_nid (void ); +erofs_off_t nid2addr (erofs_nid_t nid); +erofs_nid_t addr2nid (erofs_off_t addr); +void *erofs_init (struct fuse_conn_info *info); + +#endif
fuse/logging.c new file +81 -0
81 lines not on this page · raw message
fuse/logging.h new file +55 -0
55 lines not on this page · raw message
fuse/main.c new file +170 -0
170 lines not on this page · raw message
fuse/namei.c new file +227 -0
227 lines not on this page · raw message
fuse/namei.h new file +22 -0
@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* erofs-fuse\inode.h +* Created by Li Guifu <[email protected]> +*/ + +#ifndef __INODE_H +#define __INODE_H + +#include "erofs/internal.h" +#include "erofs_fs.h" + +int inode_init (erofs_nid_t root); +struct dcache_entry *get_cached_dentry (struct dcache_entry **parent, +const char **path); +int erofs_iget_by_path (const char *path,struct erofs_vnode *v); +int erofs_iget_by_nid (erofs_nid_t nid,struct erofs_vnode *v); +struct dcache_entry *disk_lookup (struct dcache_entry *parent,const char *name, +unsigned int name_len); +int walk_path (const char *path,erofs_nid_t *out_nid); + +#endif
fuse/open.c new file +22 -0
@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* +* erofs-fuse\open.c +* Created by Li Guifu <[email protected]> +*/ + +#include "open.h" +#include <asm-generic/errno-base.h> +#include <fuse.h> +#include <fuse_opt.h> +#include "logging.h" + +int erofs_open (const char *path,struct fuse_file_info *fi) +{ +logi ("open path=%s" , path); + +if ((fi->flags &O_ACCMODE ) !=O_RDONLY ) +return -EACCES ; + +return 0 ; +} +
fuse/open.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* erofs-fuse\open.h +* Created by Li Guifu <[email protected]> +*/ + +#ifndef __EROFS_OPEN_H +#define __EROFS_OPEN_H + +#include <fuse.h> +#include <fuse_opt.h> + +int erofs_open (const char *path,struct fuse_file_info *fi); + +#endif
fuse/read.c new file +114 -0
114 lines not on this page · raw message
fuse/read.h new file +16 -0
@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* erofs-fuse\read.h +* Created by Li Guifu <[email protected]> +*/ + +#ifndef __EROFS_READ_H +#define __EROFS_READ_H + +#include <fuse.h> +#include <fuse_opt.h> + +int erofs_read (const char *path,char *buffer,size_t size,off_t offset, +struct fuse_file_info *fi); + +#endif
fuse/readir.c new file +123 -0
123 lines not on this page · raw message
fuse/readir.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* erofs-fuse\readir.h +* Created by Li Guifu <[email protected]> +*/ + +#ifndef __EROFS_READDIR_H +#define __EROFS_READDIR_H + +#include <fuse.h> +#include <fuse_opt.h> + +int erofs_readdir (const char *path,void *buffer,fuse_fill_dir_t filler, +off_t offset,struct fuse_file_info *fi); + + +#endif
include/erofs/defs.h +3 -0
@@ -24,6 +24,9 @@#ifdef HAVE_LINUX_TYPES_H #include <linux/types.h> #endif +#ifndef UNUSED +#define UNUSED (_X) ((void) _X) +#endif /* * container_of - cast a member of a structure out to the containing structure
include/erofs/internal.h +37 -0
@@ -55,6 +55,8 @@ typedef u32 erofs_blk_t;#define blknr_to_addr (nr) ((erofs_off_t)(nr) * EROFS_BLKSIZ)#define BLK_ROUND_UP (addr) DIV_ROUND_UP(addr, EROFS_BLKSIZ) +#define IS_SLOT_ALIGN (__ADDR) (((__ADDR) % (EROFS_SLOTSIZE)) ? 0 : 1) +#define IS_BLK_ALIGN (__ADDR) (((__ADDR) % (EROFS_BLKSIZ)) ? 0 : 1)struct erofs_buffer_head ;
@@ -132,11 +134,46 @@ struct erofs_inode {#endif }; +struct erofs_vnode { +uint8_t datalayout; + +uint32_t i_size; +/* inline size in bytes */ +uint16_t inode_isize; +uint16_t xattr_isize; + +uint16_t xattr_shared_count; +char *xattr_shared_xattrs; + +erofs_blk_t raw_blkaddr; +erofs_nid_t nid; +uint32_t i_ino; + +uint16_t i_mode; +uint16_t i_uid; +uint16_t i_gid; +uint16_t i_nlink; + +/* if file is inline read inline data witch inode */ +char *idata; +}; +static inline bool is_inode_layout_compression (struct erofs_inode *inode) {return erofs_inode_is_data_compressed (inode->datalayout); } +#define __inode_advise (x, bit, bits) \ + (((x) >> (bit)) & ((1 << (bits)) - 1)) + +#define __inode_version (advise) \ + __inode_advise(advise, EROFS_I_VERSION_BIT, \ + EROFS_I_VERSION_BITS) + +#define __inode_data_mapping (advise) \ + __inode_advise(advise, EROFS_I_DATALAYOUT_BIT,\ + EROFS_I_DATALAYOUT_BITS) +#define IS_ROOT (x) ((x) == (x)->i_parent)struct erofs_dentry {
Signature · 1 line
2.24.0
PATCH 2/12 erofs-utils: fuse: support read special file
In reply to Gao Xiang via Linux-erofsFrom: Huang Jianan <[email protected]>
--- fuse/getattr.c | 1 + fuse/main.c | 1 + fuse/namei.c | 14 ++++++++++---- fuse/read.c | 26 ++++++++++++++++++++++++++ fuse/read.h | 1 + include/erofs/internal.h | 1 + 6 files changed, 40 insertions(+), 4 deletions(-)
fuse/getattr.c +1 -0
@@ -56,6 +56,7 @@ int erofs_getattr(const char *path, struct stat *stbuf) stbuf->st_blocks = stbuf->st_size /EROFS_BLKSIZ ; stbuf->st_uid =le16_to_cpu (v.i_uid); stbuf->st_gid =le16_to_cpu (v.i_gid); + stbuf->st_rdev =le32_to_cpu (v.i_rdev); stbuf->st_atime = super.build_time; stbuf->st_mtime = super.build_time; stbuf->st_ctime = super.build_time;
fuse/main.c +1 -0
@@ -113,6 +113,7 @@ static void signal_handle_sigsegv(int signal) }static struct fuse_operations erofs_ops = { + .readlink = erofs_readlink, .getattr = erofs_getattr, .readdir = erofs_readdir, .open = erofs_open,
fuse/namei.c +10 -4
@@ -11,6 +11,7 @@#include <stdio.h> #include <errno.h> #include <sys/stat.h> +#include <sys/sysmacros.h> #include "erofs/defs.h" #include "logging.h"
@@ -39,6 +40,13 @@ static uint8_t get_path_token_len(const char *path)return len; } +static inline dev_t new_decode_dev (u32 dev) +{ +unsigned major = (dev &0xfff00 ) >>8 ; +unsigned minor = (dev &0xff ) | ((dev >>12 ) &0xfff00 ); +return makedev (major, minor); +} +int erofs_iget_by_nid (erofs_nid_t nid,struct erofs_vnode *vi) {int ret;
@@ -65,13 +73,11 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)switch (vi->i_mode &S_IFMT ) {case S_IFBLK :case S_IFCHR : -/* fixme: add special devices support -* vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev)); -*/ + vi->i_rdev =new_decode_dev (le32_to_cpu (v1->i_u.rdev));break ;case S_IFIFO :case S_IFSOCK : -/*fixme: vi->i_rdev = 0; */ + vi->i_rdev =0 ;break ;case S_IFREG :case S_IFLNK :
fuse/read.c +26 -0
@@ -112,3 +112,29 @@ int erofs_read(const char *path, char *buffer, size_t size, off_t offset,return -EINVAL ; } } + +int erofs_readlink (const char *path,char *buffer,size_t size) +{ +int ret; +erofs_nid_t nid; +size_t lnksz; +struct erofs_vnode v; + + ret =walk_path (path, &nid); +if (ret) +return ret; + + ret =erofs_iget_by_nid (nid, &v); +if (ret) +return ret; + + lnksz =min ((size_t )v.i_size, size -1 ); + + ret =erofs_read (path, buffer, lnksz,0 ,NULL ); + buffer[lnksz] ='\0' ; +if (ret != (int )lnksz) +return ret; + +logi ("path:%s link=%s size=%llu" , path, buffer, lnksz); +return 0 ; +}
fuse/read.h +1 -0
@@ -12,5 +12,6 @@int erofs_read (const char *path,char *buffer,size_t size,off_t offset,struct fuse_file_info *fi); +int erofs_readlink (const char *path,char *buffer,size_t size);#endif
include/erofs/internal.h +1 -0
@@ -153,6 +153,7 @@ struct erofs_vnode {uint16_t i_uid;uint16_t i_gid;uint16_t i_nlink; +uint32_t i_rdev;/* if file is inline read inline data witch inode */ char *idata;
Signature · 1 line
2.24.0
PATCH 3/12 erofs-utils: fuse: support read compressed file
In reply to Gao Xiang via Linux-erofsFrom: Huang Jianan <[email protected]>
--- fuse/Makefile.am | 8 +- fuse/decompress.c | 86 ++++++++ fuse/decompress.h | 37 ++++ fuse/dentry.h | 5 +- fuse/init.c | 22 ++- fuse/init.h | 2 + fuse/namei.c | 11 +- fuse/read.c | 73 ++++++- fuse/zmap.c | 416 +++++++++++++++++++++++++++++++++++++++ include/erofs/defs.h | 13 ++ include/erofs/internal.h | 43 +++- include/erofs_fs.h | 4 + 12 files changed, 711 insertions(+), 9 deletions(-) create mode 100644 fuse/decompress.c create mode 100644 fuse/decompress.h create mode 100644 fuse/zmap.c
fuse/Makefile.am +6 -2
@@ -3,12 +3,16 @@ AUTOMAKE_OPTIONS = foreign bin_PROGRAMS = erofsfuse -erofsfuse_SOURCES = main.c dentry.c getattr.c logging.c namei.c read.c disk_io.c init.c open.c readir.c +erofsfuse_SOURCES = main.c dentry.c getattr.c logging.c namei.c read.c disk_io.c init.c open.c readir.c zmap.c +if ENABLE_LZ4 +erofsfuse_SOURCES += decompress.c +endif erofsfuse_CFLAGS = -Wall -Werror -Wextra \ + -Wno-implicit-fallthrough \ -I$(top_srcdir)/include \ $(shell pkg-config fuse --cflags) \ -DFUSE_USE_VERSION=26 \ -std=gnu99 LDFLAGS += $(shell pkg-config fuse --libs) -erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl +erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl ${LZ4_LIBS}
fuse/decompress.c new file +86 -0
86 lines not on this page · raw message
fuse/decompress.h new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd. +* Created by Huang Jianan <[email protected]> +*/ + +#ifndef __EROFS_DECOMPRESS_H +#define __EROFS_DECOMPRESS_H + +#include "erofs/internal.h" + +enum { +Z_EROFS_COMPRESSION_SHIFTED =Z_EROFS_COMPRESSION_MAX , +Z_EROFS_COMPRESSION_RUNTIME_MAX +}; + +struct z_erofs_decompress_req { +char *in, *out; + +size_t ofs_out, ofs_head; +unsigned int inputsize, outputsize; + +/* indicate the algorithm will be used for decompression */ +unsigned int alg; +bool partial_decoding; +}; + +#ifdef LZ4_ENABLED +int z_erofs_decompress (struct z_erofs_decompress_req *rq); +#else +int z_erofs_decompress (struct z_erofs_decompress_req *rq) +{ +return -EOPNOTSUPP ; +} +#endif + +#endif
fuse/dentry.h +3 -2
@@ -10,10 +10,11 @@#include <stdint.h> #include "erofs/internal.h" +/* fixme: Deal with names that exceed the allocated size */ #ifdef __64BITS -#define DCACHE_ENTRY_NAME_LEN 40 +#define DCACHE_ENTRY_NAME_LEN EROFS_NAME_LEN#else -#define DCACHE_ENTRY_NAME_LEN 48 +#define DCACHE_ENTRY_NAME_LEN EROFS_NAME_LEN#endif /* This struct declares a node of a k-tree. Every node has a pointer to one of
fuse/init.c +21 -1
@@ -17,7 +17,23 @@struct erofs_super_block super; -static struct erofs_super_block *sbk = &super; +struct erofs_super_block *sbk = &super; + +static bool check_layout_compatibility (struct erofs_super_block *sb, +struct erofs_super_block *dsb) +{ +const unsigned int feature =le32_to_cpu (dsb->feature_incompat); + + sb->feature_incompat = feature; + +/* check if current kernel meets all mandatory requirements */ +if (feature & (~EROFS_ALL_FEATURE_INCOMPAT )) { +loge ("unidentified incompatible feature %x, please upgrade kernel version" , + feature & ~EROFS_ALL_FEATURE_INCOMPAT ); +return false; + } +return true; +}int erofs_init_super (void ) {
@@ -40,6 +56,9 @@ int erofs_init_super(void)return -EINVAL ; } +if (!check_layout_compatibility (sbk, sb)) +return -EINVAL ; + sbk->checksum =le32_to_cpu (sb->checksum); sbk->feature_compat =le32_to_cpu (sb->feature_compat); sbk->blkszbits = sb->blkszbits;
@@ -56,6 +75,7 @@ int erofs_init_super(void) sbk->root_nid =le16_to_cpu (sb->root_nid);logp ("%-15s:0x%X" ,STR (magic),SUPER_MEM (magic)); +logp ("%-15s:0x%X" ,STR (feature_incompat),SUPER_MEM (feature_incompat));logp ("%-15s:0x%X" ,STR (feature_compat),SUPER_MEM (feature_compat));logp ("%-15s:%u" ,STR (blkszbits),SUPER_MEM (blkszbits));logp ("%-15s:%u" ,STR (root_nid),SUPER_MEM (root_nid));
fuse/init.h +2 -0
@@ -13,6 +13,8 @@#define BOOT_SECTOR_SIZE 0x400 +extern struct erofs_super_block *sbk; +int erofs_init_super (void );erofs_nid_t erofs_get_root_nid (void );erofs_off_t nid2addr (erofs_nid_t nid);
fuse/namei.c +10 -1
@@ -49,7 +49,7 @@ static inline dev_t new_decode_dev(u32 dev)int erofs_iget_by_nid (erofs_nid_t nid,struct erofs_vnode *vi) { -int ret; +int ret, ifmt;char buf[EROFS_BLKSIZ ];struct erofs_inode_compact *v1;const erofs_off_t addr =nid2addr (nid);
@@ -60,6 +60,11 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)return -EIO ; v1 = (struct erofs_inode_compact *)buf; +/* fixme: support extended inode */ + ifmt =le16_to_cpu (v1->i_format); +if (__inode_version (ifmt) !=EROFS_INODE_LAYOUT_COMPACT ) +return -EOPNOTSUPP ; + vi->datalayout =__inode_data_mapping (le16_to_cpu (v1->i_format)); vi->inode_isize =sizeof (struct erofs_inode_compact ); vi->xattr_isize =erofs_xattr_ibody_size (v1->i_xattr_icount);
@@ -88,6 +93,10 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)return -EIO ; } + vi->z_inited = false; +if (erofs_inode_is_data_compressed (vi->datalayout)) +z_erofs_fill_inode (vi); +return 0 ; }
fuse/read.c +71 -2
95 lines not on this page · raw message
fuse/zmap.c new file +416 -0
416 lines not on this page · raw message
include/erofs/defs.h +13 -0
@@ -173,5 +173,18 @@ typedef int64_t s64;#define __maybe_unused __attribute__((__unused__))#endif +struct __una_u32 {u32 x; } __packed; + +static inline u32 __get_unaligned_cpu32 (const void *p) +{ +const struct __una_u32 *ptr = (const struct __una_u32 *)p; +return ptr->x; +} + +static inline u32 get_unaligned_le32 (const void *p) +{ +return __get_unaligned_cpu32 ((const u8 *)p); +} +#endif
include/erofs/internal.h +42 -1
66 lines not on this page · raw message
include/erofs_fs.h +4 -0
@@ -279,6 +279,10 @@ struct z_erofs_vle_decompressed_index { } di_u; }; +#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN (size) \ + (round_up(size, sizeof(struct z_erofs_vle_decompressed_index)) + \ + sizeof(struct z_erofs_map_header) + Z_EROFS_VLE_LEGACY_HEADER_PADDING) +#define Z_EROFS_VLE_EXTENT_ALIGN (size) round_up(size, \ sizeof(struct z_erofs_vle_decompressed_index))
Signature · 1 line
2.24.0
PATCH 4/12 erofs-utils: fuse: adjust larger extent handling
In reply to Gao Xiang via Linux-erofsso more easy to understand.
--- fuse/read.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
fuse/read.c +9 -4
@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer, Z_EROFS_COMPRESSION_LZ4 :Z_EROFS_COMPRESSION_SHIFTED ; -if (end >= map.m_la + map.m_llen) { - count = map.m_llen; - partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED ); - }else { +/* +* trim to the needed size if the returned extent is quite +* larger than requested, and set up partial flag as well. +*/ +if (end < map.m_la + map.m_llen) { count = end - map.m_la; partial = true; + }else { +ASSERT (end == map.m_la + map_m_llen); + count = map.m_llen; + partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED ); }if ((off_t )map.m_la < offset) {
Signature · 1 line
2.24.0
回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
Hi, Gao Xiang,
________________________________
发件人: Gao Xiang<mailto:[email protected]>
发送时间: 2020-10-17 13:16
收件人: linux-erofs<mailto:[email protected]>
抄送: Huang Jianan<mailto:[email protected]>; Li Guifu<mailto:[email protected]>; Li Guifu<mailto:[email protected]>; Chao Yu<mailto:[email protected]>; Guo Weichao<mailto:[email protected]>; Zhang Shiming<mailto:[email protected]>; Gao Xiang<mailto:[email protected]> 主题: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
so more easy to understand.
---fuse/read.c | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
fuse/read.c +0 -0
@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
Z_EROFS_COMPRESSION_LZ4 :Z_EROFS_COMPRESSION_SHIFTED;
- if (end >= map.m_la + map.m_llen) {
- count = map.m_llen;
- partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
- } else {
+ /*
+ * trim to the needed size if the returned extent is quite
+ * larger than requested, and set up partial flag as well.
+ */
+ if (end < map.m_la + map.m_llen) {
count = end - map.m_la;
partial = true;
+ } else {
+ ASSERT(end == map.m_la + map_m_llen);
I think you mean map.m_llen intesad of map_m_llen.
Besides, I don't understand why add ASSERT here.
I think this condition will be true if offset+size is exactly the end of a compressed block?
+ count = map.m_llen;
+ partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED); }
if ((off_t)map.m_la < offset) {
--
2.24.0
Thanks,
Jianan
________________________________
OPPO
本电子邮件及其附件含有OPPO公司的保密信息,仅限于邮件指明的收件人使用(包含个人及群组)。禁止任何人在未经授权的情况下以任何形式使用。如果您错收了本邮件,请立即以电子邮件通知发件人并删除本邮件及其附件。
This e-mail and its attachments contain confidential information from OPPO, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!
Re: 回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
Hi Jianan,
Quoted 38 lines · On Mon, Oct 19, 2020 at 11:13:42AM +0800, [email protected] wrote:
Hi, Gao Xiang锟斤拷
________________________________锟斤拷锟斤拷锟剿o拷 Gao Xiang<mailto:[email protected]>
锟斤拷锟斤拷时锟戒: 2020-10-17 13:16
锟秸硷拷锟剿o拷 linux-erofs<mailto:[email protected]>
锟斤拷锟酵o拷 Huang Jianan<mailto:[email protected]>; Li Guifu<mailto:[email protected]>; Li Guifu<mailto:[email protected]>; Chao Yu<mailto:[email protected]>; Guo Weichao<mailto:[email protected]>; Zhang Shiming<mailto:[email protected]>; Gao Xiang<mailto:[email protected]> 锟斤拷锟解: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
so more easy to understand.[ let's fold in to the original patch. ]Cc Huang Jianan <[email protected]>Signed-off-by Gao Xiang <[email protected]>---
fuse/read.c| 13 +++++++++----1 file changed, 9 insertions(+), 4 deletions(-)fuse/read.c +0
-0@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
Z_EROFS_COMPRESSION_LZ4:Z_EROFS_COMPRESSION_SHIFTED;
- if (end >= map.m_la + map.m_llen) {
- count = map.m_llen;
- partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED);
- } else {
+ /*
+ * trim to the needed size if the returned extent is quite
+ * larger than requested, and set up partial flag as well.
+ */
+ if (end < map.m_la + map.m_llen) {
count = end - map.m_la;
partial = true;
+ } else {
+ASSERT(end == map.m_la + map_m_llen);
Quoted 3 lines
I think you mean map.m_llen intesad of
map_m_llen.
Besides, I don't understand why add ASSERT here.
I think this condition will be true if offset+size is exactly the end of a compressed block?
Thanks for your question.
The idea is that we requested the extent with
map.m_la = end - 1; ret = z_erofs_map_blocks_iter(vnode, &map); if (ret) return ret;
so the extent must include "end - 1", so it's impossible that "end > map.m_la + map.m_llen" (invalid return).
or the entire extent would be holed extent, anyway, that is another extent rather than a data extent.
(BTW, the up-to-date commits is at
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=wip/experimental_fuse
kindly check out them as well :) )
Thanks,
Gao Xiang
Quoted 10 lines
+ count = map.m_llen;
+ partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED); }
if ((off_t)map.m_la < offset) {
--
2.24.0Thanks,
Jianan
回复: Re: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
Hi, Gao Xiang,
Thanks for you answer, I understand now.
In addition, I sent a patch to fix the clerical error here.
It seems that the compilation won't be affected under the default configuration.
Thanks,
Jianan
--------------
Huang Jianan
Quoted 61 lines
Hi Jianan,
Quoted 38 lines · On Mon, Oct 19, 2020 at 11:13:42AM +0800, [email protected] wrote:
Hi, Gao Xiang锟斤拷
________________________________锟斤拷锟斤拷锟剿o拷 Gao Xiang<mailto:[email protected]>
锟斤拷锟斤拷时锟戒: 2020-10-17 13:16
锟秸硷拷锟剿o拷 linux-erofs<mailto:[email protected]>
锟斤拷锟酵o拷 Huang Jianan<mailto:[email protected]>; Li Guifu<mailto:[email protected]>; Li Guifu<mailto:[email protected]>; Chao Yu<mailto:[email protected]>; Guo Weichao<mailto:[email protected]>; Zhang Shiming<mailto:[email protected]>; Gao Xiang<mailto:[email protected]> 锟斤拷锟解: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
so more easy to understand.[ let's fold in to the original patch. ]Cc Huang Jianan <[email protected]>Signed-off-by Gao Xiang <[email protected]>---
fuse/read.c| 13 +++++++++----1 file changed, 9 insertions(+), 4 deletions(-)fuse/read.c +0
-0@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
Z_EROFS_COMPRESSION_LZ4:Z_EROFS_COMPRESSION_SHIFTED;
- if (end >= map.m_la + map.m_llen) {
- count = map.m_llen;
- partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED);
- } else {
+ /*
+ * trim to the needed size if the returned extent is quite
+ * larger than requested, and set up partial flag as well.
+ */
+ if (end < map.m_la + map.m_llen) {
count = end - map.m_la;
partial = true;
+ } else {
+ASSERT(end == map.m_la + map_m_llen);Quoted 3 lines
I think you mean map.m_llen intesad of
map_m_llen.
Besides, I don't understand why add ASSERT here.
I think this condition will be true if offset+size is exactly the end of a compressed block?Thanks for your question.
The idea is that we requested the extent withmap.m_la = end - 1; ret = z_erofs_map_blocks_iter(vnode, &map); if (ret) return ret;so the extent must include "end - 1", so it's impossible that "end > map.m_la + map.m_llen" (invalid return).
or the entire extent would be holed extent, anyway, that is another extent rather than a data extent.
Quoted 19 lines
(BTW, the up-to-date commits is at
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=wip/experimental_fuse
kindly check out them as well :) )Thanks,
Gao XiangQuoted 10 lines
+ count = map.m_llen;
+ partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED); }
if ((off_t)map.m_la < offset) {
--
2.24.0Thanks,
Jianan
________________________________
OPPO
本电子邮件及其附件含有OPPO公司的保密信息,仅限于邮件指明的收件人使用(包含个人及群组)。禁止任何人在未经授权的情况下以任何形式使用。如果您错收了本邮件,请立即以电子邮件通知发件人并删除本邮件及其附件。
This e-mail and its attachments contain confidential information from OPPO, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!
回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
In reply to Gao Xiang via Linux-erofsHi, Gao Xiang,
Sorry for the format of the previous email.
Quoted 28 lines
so more easy to understand.
[ let's fold in to the original patch. ]Cc Huang Jianan <[email protected]>Signed-off-by Gao Xiang <[email protected]>---
fuse/read.c| 13 +++++++++----1 file changed, 9 insertions(+), 4 deletions(-)fuse/read.c +0
-0@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
Z_EROFS_COMPRESSION_LZ4:Z_EROFS_COMPRESSION_SHIFTED;fuse/read.c +5
-4-if (end >= map.m_la + map.m_llen) { - count = map.m_llen; - partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED ); - }else { +/* +* trim to the needed size if the returned extent is quite +* larger than requested, and set up partial flag as well. +*/ +if (end < map.m_la + map.m_llen) {count = end - map.m_la;
partial = true;
fuse/read.c size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer, · line 123
Quoted 2 lines
fuse/read.c +2
-0+ }else { +ASSERT (end == map.m_la + map_m_llen);
I think you mean map.m_llen intesad of map_m_llen.
Besides, I don't understand why add ASSERT here.
I think this condition will be true if offset+size is exactly the end of a compressed block?
Quoted 8 lines
fuse/read.c +2
-0+ count = map.m_llen; + partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED );}
if ((off_t)map.m_la < offset) {
--
2.24.0
Thanks,
Jianan
________________________________
OPPO
本电子邮件及其附件含有OPPO公司的保密信息,仅限于邮件指明的收件人使用(包含个人及群组)。禁止任何人在未经授权的情况下以任何形式使用。如果您错收了本邮件,请立即以电子邮件通知发件人并删除本邮件及其附件。
This e-mail and its attachments contain confidential information from OPPO, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!
Re: 回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
Hi Jianan,
Quoted 2 lines · Huang Jianan
Hi, Gao Xiang,
Sorry for the format of the previous email.
I've replied the previous email at
https://lore.kernel.org/r/20201019033251.GA29138@hsiangkao-HP-ZHAN-66-Pro-G1
As for the email format, I think you can give a try to use mutt or Thunderbird (assume your company PC is in MS Windows) , which is more widely used in open-source communities :)
Thanks,
Gao Xiang
Quoted 1 line
PATCH 5/12 erofs-utils: fuse: use proper expression about inode size
In reply to Gao Xiang via Linux-erofs--- fuse/namei.c | 4 ++-- fuse/read.c | 2 +- fuse/readir.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
fuse/namei.c +2 -2
@@ -172,8 +172,8 @@ struct dcache_entry *disk_lookup(struct dcache_entry *parent, const char *name,if (v.datalayout ==EROFS_INODE_FLAT_INLINE ) {uint32_t dir_off =erofs_blkoff (dirsize); -off_t dir_addr =nid2addr (dcache_get_nid (parent)) - +sizeof (struct erofs_inode_compact ) + v.xattr_isize; +off_t dir_addr =nid2addr (dcache_get_nid (parent)) + + v.inode_isize + v.xattr_isize;memset (buf,0 ,sizeof (buf)); ret =dev_read (buf, dir_off, dir_addr);
fuse/read.c +1 -1
@@ -65,7 +65,7 @@ size_t erofs_read_data_inline(struct erofs_vnode *vnode, char *buffer,if (!suminline) goto finished; - addr =nid2addr (vnode->nid) +sizeof (struct erofs_inode_compact ) + addr =nid2addr (vnode->nid) + vnode->inode_isize + + vnode->xattr_isize; ret =dev_read (buffer + rdsz, suminline, addr);if (ret <0 || (size_t )ret != suminline)
fuse/readir.c +1 -1
@@ -108,7 +108,7 @@ int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,if (v.datalayout ==EROFS_INODE_FLAT_INLINE ) {off_t addr; - addr =nid2addr (nid) +sizeof (struct erofs_inode_compact ) + addr =nid2addr (nid) + v.inode_isize + v.xattr_isize;memset (dirsbuf,0 ,sizeof (dirsbuf));
Signature · 1 line
2.24.0
PATCH 6/12 erofs-utils: fuse: drop ofs_out
In reply to Gao Xiang via Linux-erofs--- fuse/decompress.c | 7 +++---- fuse/decompress.h | 2 +- fuse/read.c | 3 +-- 3 files changed, 5 insertions(+), 7 deletions(-)
fuse/decompress.c +3 -4
@@ -15,7 +15,7 @@static int z_erofs_shifted_transform (struct z_erofs_decompress_req *rq) { -char *dest = rq->out + rq->ofs_out; +char *dest = rq->out;char *src = rq->in + rq->ofs_head;memcpy (dest, src, rq->outputsize - rq->ofs_head);
@@ -26,7 +26,7 @@ static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq)static int z_erofs_decompress_generic (struct z_erofs_decompress_req *rq) {int ret =0 ; -char *dest = rq->out + rq->ofs_out; +char *dest = rq->out;char *src = rq->in;char *buff =NULL ;bool support_0padding = false;
@@ -66,8 +66,7 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)if (rq->ofs_head) { src = dest + rq->ofs_head; - dest = rq->out + rq->ofs_out; -memcpy (dest, src, rq->outputsize - rq->ofs_head); +memcpy (rq->out, src, rq->outputsize - rq->ofs_head); } out:
fuse/decompress.h +1 -1
@@ -17,7 +17,7 @@ enum {struct z_erofs_decompress_req {char *in, *out; -size_t ofs_out, ofs_head; +size_t ofs_head;unsigned int inputsize, outputsize;/* indicate the algorithm will be used for decompression */
fuse/read.c +1 -2
@@ -135,8 +135,7 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer, ret = z_erofs_decompress(&(struct z_erofs_decompress_req) { .in = raw, - .out = buffer, - .ofs_out = sum, + .out = buffer + sum, .ofs_head = ofs, .inputsize = EROFS_BLKSIZ, .outputsize = count,
Signature · 1 line
2.24.0
PATCH 7/12 erofs-utils: fuse: refuse a undefined shifted cluster behavior
In reply to Gao Xiang via Linux-erofs--- fuse/decompress.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
fuse/decompress.c +4 -1
@@ -18,8 +18,11 @@ static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq)char *dest = rq->out;char *src = rq->in + rq->ofs_head; -memcpy (dest, src, rq->outputsize - rq->ofs_head); +if (rq->inputsize !=EROFS_BLKSIZ ) +return -EFSCORRUPTED ; +DBG_BUGON (rq->outputsize >EROFS_BLKSIZ ); +memcpy (dest, src, rq->outputsize - rq->ofs_head);return 0 ; }
Signature · 1 line
2.24.0
PATCH 8/12 erofs-utils: fuse: drop z_erofs_shifted_transform()
In reply to Gao Xiang via Linux-erofsclean up as well.
--- fuse/decompress.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-)
fuse/decompress.c +11 -15
@@ -13,19 +13,6 @@#include "logging.h" #include "init.h" -static int z_erofs_shifted_transform (struct z_erofs_decompress_req *rq) -{ -char *dest = rq->out; -char *src = rq->in + rq->ofs_head; - -if (rq->inputsize !=EROFS_BLKSIZ ) -return -EFSCORRUPTED ; - -DBG_BUGON (rq->outputsize >EROFS_BLKSIZ ); -memcpy (dest, src, rq->outputsize - rq->ofs_head); -return 0 ; -} -static int z_erofs_decompress_generic (struct z_erofs_decompress_req *rq) {int ret =0 ;
@@ -81,8 +68,17 @@ out:int z_erofs_decompress (struct z_erofs_decompress_req *rq) { -if (rq->alg ==Z_EROFS_COMPRESSION_SHIFTED ) -return z_erofs_shifted_transform (rq); +if (rq->alg ==Z_EROFS_COMPRESSION_SHIFTED ) { +if (rq->inputsize !=EROFS_BLKSIZ ) +return -EFSCORRUPTED ; + +DBG_BUGON (rq->outputsize >EROFS_BLKSIZ ); +DBG_BUGON (rq->outputsize < rq->ofs_head); + +memcpy (rq->out, rq->in + rq->ofs_head, + rq->outputsize - rq->ofs_head); +return 0 ; + }return z_erofs_decompress_generic (rq); }
Signature · 1 line
2.24.0
PATCH 9/12 erofs-utils: fuse: rename ofs_head and outputsize
In reply to Gao Xiang via Linux-erofs--- fuse/decompress.c | 27 +++++++++++++-------------- fuse/decompress.h | 8 ++++++-- fuse/read.c | 4 ++-- 3 files changed, 21 insertions(+), 18 deletions(-)
fuse/decompress.c +13 -14
@@ -33,8 +33,8 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)return -EIO ; } -if (rq->ofs_head) { - buff =malloc (rq->outputsize); +if (rq->decodedskip) { + buff =malloc (rq->decodedlength);if (!buff)return -ENOMEM ; dest = buff;
@@ -42,22 +42,21 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)if (rq->partial_decoding || !support_0padding) ret =LZ4_decompress_safe_partial (src + inputmargin, dest, - rq->inputsize - inputmargin, - rq->outputsize, rq->outputsize); + rq->inputsize - inputmargin, + rq->decodedlength, rq->decodedlength);else ret =LZ4_decompress_safe (src + inputmargin, dest, rq->inputsize - inputmargin, - rq->outputsize); + rq->decodedlength); -if (ret != (int )rq->outputsize) { +if (ret != (int )rq->decodedlength) { ret = -EIO ; goto out; } -if (rq->ofs_head) { - src = dest + rq->ofs_head; -memcpy (rq->out, src, rq->outputsize - rq->ofs_head); - } +if (rq->decodedskip) +memcpy (rq->out, dest + rq->decodedskip, + rq->decodedlength - rq->decodedskip); out:if (buff)
@@ -72,11 +71,11 @@ int z_erofs_decompress(struct z_erofs_decompress_req *rq)if (rq->inputsize !=EROFS_BLKSIZ )return -EFSCORRUPTED ; -DBG_BUGON (rq->outputsize >EROFS_BLKSIZ ); -DBG_BUGON (rq->outputsize < rq->ofs_head); +DBG_BUGON (rq->decodedlength >EROFS_BLKSIZ ); +DBG_BUGON (rq->decodedlength < rq->decodedskip); -memcpy (rq->out, rq->in + rq->ofs_head, - rq->outputsize - rq->ofs_head); +memcpy (rq->out, rq->in + rq->decodedskip, + rq->decodedlength - rq->decodedskip);return 0 ; }
fuse/decompress.h +6 -2
@@ -17,8 +17,12 @@ enum {struct z_erofs_decompress_req {char *in, *out; -size_t ofs_head; -unsigned int inputsize, outputsize; +/* +* initial decompressed bytes that need to be skipped +* when finally copying to output buffer +*/ +unsigned int decodedskip; +unsigned int inputsize, decodedlength;/* indicate the algorithm will be used for decompression */ unsigned int alg;
fuse/read.c +2 -2
@@ -136,9 +136,9 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer, ret =z_erofs_decompress (&(struct z_erofs_decompress_req ) { .in = raw, .out = buffer + sum, - .ofs_head = ofs, + .decodedskip = ofs, .inputsize =EROFS_BLKSIZ , - .outputsize = count, + .decodedlength = count, .alg = algorithmformat, .partial_decoding = partial });
Signature · 1 line
2.24.0
PATCH 10/12 erofs-utils: fuse: cleanup erofs_read_data_compression()
In reply to Gao Xiang via Linux-erofs--- fuse/read.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-)
fuse/read.c +16 -16
@@ -80,10 +80,10 @@ finished: }size_t erofs_read_data_compression (struct erofs_vnode *vnode,char *buffer, -size_t size,off_t offset) +erofs_off_t size,erofs_off_t offset) {int ret; -size_t end, count, ofs, sum = size; +erofs_off_t end, length, skip;struct erofs_map_blocks map = { .index =UINT_MAX , };
@@ -91,8 +91,8 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,unsigned int algorithmformat;char raw[EROFS_BLKSIZ ]; -while (sum) { - end = offset + sum; + end = offset + size; +while (end > offset) { map.m_la = end -1 ; ret =z_erofs_map_blocks_iter (vnode, &map);
@@ -100,7 +100,7 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,return ret;if (!(map.m_flags &EROFS_MAP_MAPPED )) { - sum -= map.m_llen; + end = map.m_la;continue ; }
@@ -117,28 +117,28 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,* larger than requested, and set up partial flag as well. */ if (end < map.m_la + map.m_llen) { - count = end - map.m_la; + length = end - map.m_la; partial = true; }else {ASSERT (end == map.m_la + map_m_llen); - count = map.m_llen; + length = map.m_llen; partial = !(map.m_flags &EROFS_MAP_FULL_MAPPED ); } -if ((off_t )map.m_la < offset) { - ofs = offset - map.m_la; - sum =0 ; +if (map.m_la < offset) { + skip = offset - map.m_la; + end = offset; }else { - ofs =0 ; - sum -= count; + skip =0 ; + end = map.m_la; } ret =z_erofs_decompress (&(struct z_erofs_decompress_req ) { .in = raw, - .out = buffer + sum, - .decodedskip = ofs, - .inputsize =EROFS_BLKSIZ , - .decodedlength = count, + .out = buffer + end - offset, + .decodedskip = skip, + .inputsize = map.m_plen, + .decodedlength = length, .alg = algorithmformat, .partial_decoding = partial });
Signature · 1 line
2.24.0
PATCH 11/12 erofs-utils: fuse: move up mpage in struct erofs_map_blocks
In reply to Gao Xiang via Linux-erofs--- include/erofs/internal.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/erofs/internal.h +2 -2
@@ -232,12 +232,12 @@ enum {#define EROFS_MAP_FULL_MAPPED (1 << BH_FullMapped)struct erofs_map_blocks { +char mpage[EROFS_BLKSIZ ]; +erofs_off_t m_pa, m_la;u64 m_plen, m_llen;unsigned int m_flags; - -char mpage[EROFS_BLKSIZ ];erofs_blk_t index; };
Signature · 1 line
2.24.0
PATCH 12/12 erofs-utils: fuse: fix up source headers
In reply to Gao Xiang via Linux-erofsfix up weird paths and relicense zmap.c
--- fuse/decompress.c | 5 +++-- fuse/decompress.h | 3 ++- fuse/dentry.c | 4 ++-- fuse/dentry.h | 4 ++-- fuse/disk_io.c | 4 ++-- fuse/disk_io.h | 4 ++-- fuse/getattr.c | 4 ++-- fuse/getattr.h | 4 ++-- fuse/init.c | 4 ++-- fuse/init.h | 4 ++-- fuse/logging.c | 4 ++-- fuse/logging.h | 4 ++-- fuse/main.c | 4 ++-- fuse/namei.c | 4 ++-- fuse/namei.h | 4 ++-- fuse/open.c | 4 ++-- fuse/open.h | 4 ++-- fuse/read.c | 5 +++-- fuse/read.h | 4 ++-- fuse/readir.c | 4 ++-- fuse/readir.h | 4 ++-- fuse/zmap.c | 7 ++++--- 22 files changed, 48 insertions(+), 44 deletions(-)
fuse/decompress.c +3 -2
@@ -1,9 +1,10 @@ -/* SPDX-License-Identifier: GPL-2.0+ */ +// SPDX-License-Identifier: GPL-2.0+ /* +* erofs-utils/fuse/decompress.c +* * Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd. * Created by Huang Jianan <[email protected]> */ -#include <stdlib.h> #include <lz4.h>
fuse/decompress.h +2 -1
@@ -1,9 +1,10 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* +* erofs-utils/fuse/decompress.h +* * Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd. * Created by Huang Jianan <[email protected]> */ -#ifndef __EROFS_DECOMPRESS_H#define __EROFS_DECOMPRESS_H
fuse/dentry.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\dentry.c +* erofs-utils/fuse/dentry.c +* * Created by Li Guifu <[email protected]> */ -#include "dentry.h" #include "erofs/internal.h" #include "logging.h"
fuse/dentry.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\dentry.h +* erofs-utils/fuse/dentry.h +* * Created by Li Guifu <[email protected]> */ -#ifndef _EROFS_DENTRY_H#define _EROFS_DENTRY_H
fuse/disk_io.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\disk_io.c +* erofs-utils/fuse/disk_io.c +* * Created by Li Guifu <[email protected]> */ -#define _XOPEN_SOURCE 500#include "disk_io.h"
fuse/disk_io.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\disk_io.h +* erofs-utils/fuse/disk_io.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __DISK_IO_H#define __DISK_IO_H
fuse/getattr.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\getattr.c +* erofs-utils/fuse/getattr.c +* * Created by Li Guifu <[email protected]> */ -#include "getattr.h" #include <sys/types.h>
fuse/getattr.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\getattr.h +* erofs-utils/fuse/getattr.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __EROFS_GETATTR_H#define __EROFS_GETATTR_H
fuse/init.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\init.c +* erofs-utils/fuse/init.c +* * Created by Li Guifu <[email protected]> */ -#include "init.h" #include <string.h> #include <asm-generic/errno-base.h>
fuse/init.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\init.h +* erofs-utils/fuse/init.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __EROFS_INIT_H#define __EROFS_INIT_H
fuse/logging.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\logging.c +* erofs-utils/fuse/logging.c +* * Created by Li Guifu <[email protected]> */ -#include "logging.h" #include <stdio.h>
fuse/logging.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\logging.h +* erofs-utils/fuse/logging.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __LOGGING_H#define __LOGGING_H
fuse/main.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\main.c +* erofs-utils/fuse/main.c +* * Created by Li Guifu <[email protected]> */ -#include <stdio.h> #include <stdlib.h> #include <string.h>
fuse/namei.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\namei.c +* erofs-utils/fuse/namei.c +* * Created by Li Guifu <[email protected]> */ -#include "namei.h" #include <linux/kdev_t.h> #include <sys/types.h>
fuse/namei.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\inode.h +* erofs-utils/fuse/inode.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __INODE_H#define __INODE_H
fuse/open.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\open.c +* erofs-utils/fuse/open.c +* * Created by Li Guifu <[email protected]> */ -#include "open.h" #include <asm-generic/errno-base.h> #include <fuse.h>
fuse/open.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\open.h +* erofs-utils/fuse/open.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __EROFS_OPEN_H#define __EROFS_OPEN_H
fuse/read.c +3 -2
@@ -1,9 +1,10 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\read.c +* erofs-utils/fuse/read.c +* * Created by Li Guifu <[email protected]> +* Compression support by Huang Jianan <[email protected]> */ -#include "read.h" #include <errno.h> #include <linux/fs.h>
fuse/read.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\read.h +* erofs-utils/fuse/read.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __EROFS_READ_H#define __EROFS_READ_H
fuse/readir.c +2 -2
@@ -1,9 +1,9 @@// SPDX-License-Identifier: GPL-2.0+ /* -* erofs-fuse\readir.c +* erofs-utils/fuse/readir.c +* * Created by Li Guifu <[email protected]> */ -#include "readir.h" #include <errno.h> #include <linux/fs.h>
fuse/readir.h +2 -2
@@ -1,9 +1,9 @@/* SPDX-License-Identifier: GPL-2.0+ */ /* -* erofs-fuse\readir.h +* erofs-utils/fuse/readir.h +* * Created by Li Guifu <[email protected]> */ -#ifndef __EROFS_READDIR_H#define __EROFS_READDIR_H
fuse/zmap.c +4 -3
@@ -1,13 +1,14 @@ -// SPDX-License-Identifier: GPL-2.0-only +// SPDX-License-Identifier: GPL-2.0+ /* -* Many parts of codes are copied from Linux kernel/fs/erofs. +* erofs-utils/fuse/zmap.c +* +* (a large amount of code was adapted from Linux kernel. ) * * Copyright (C) 2018-2019 HUAWEI, Inc. * https://www.huawei.com/ * Created by Gao Xiang <[email protected]> * Modified by Huang Jianan <[email protected]> */ -#include "init.h" #include "disk_io.h" #include "logging.h"
Signature · 1 line
2.24.0